Deep learning has been widely used for protein engineering. However, it is limited by the lack of sufficient experimental data to train an accurate model for predicting the functional fitness of high-order mutants. Here, we develop SESNet, a supervised deep-learning model to predict the fitness for protein mutants by leveraging both sequence and structure information, and exploiting attention mechanism. Our model integrates local evolutionary context from homologous sequences, the global evolutionary context encoding rich semantic from the universal protein sequence space and the structure information accounting for the microenvironment around each residue in a protein. We show that SESNet outperforms state-of-the-art models for predicting the sequence-function relationship on 26 deep mutational scanning datasets. More importantly, we propose a data augmentation strategy by leveraging the data from unsupervised models to pre-train our model. After that, our model can achieve strikingly high accuracy in prediction of the fitness of protein mutants, especially for the higher order variants (> 4 mutation sites), when finetuned by using only a small number of experimental mutation data (<50). The strategy proposed is of great practical value as the required experimental effort, i.e., producing a few tens of experimental mutation data on a given protein, is generally affordable by an ordinary biochemical group and can be applied on almost any protein.

本文回顾了AIM 2022上压缩图像和视频超级分辨率的挑战。这项挑战包括两条曲目。轨道1的目标是压缩图像的超分辨率，轨迹〜2靶向压缩视频的超分辨率。在轨道1中，我们使用流行的数据集DIV2K作为培训，验证和测试集。在轨道2中，我们提出了LDV 3.0数据集，其中包含365个视频，包括LDV 2.0数据集（335个视频）和30个其他视频。在这一挑战中，有12支球队和2支球队分别提交了赛道1和赛道2的最终结果。所提出的方法和解决方案衡量了压缩图像和视频上超分辨率的最先进。提出的LDV 3.0数据集可在https://github.com/renyang-home/ldv_dataset上找到。此挑战的首页是在https://github.com/renyang-home/aim22_compresssr。

估计看不见对象的6D姿势对许多现实世界应用非常有需求。但是，当前的最新姿势估计方法只能处理以前训练的对象。在本文中，我们提出了一项新任务，以使算法能够估计测试过程中新颖对象的6D姿势估计。我们收集一个具有真实图像和合成图像的数据集，并且在测试集中最多可见48个看不见的对象。同时，我们提出了一个名为infimum Add（IADD）的新指标，这是对具有不同类型姿势歧义的对象的不变测量。还提供了针对此任务的两个阶段基线解决方案。通过训练端到端的3D对应网络，我们的方法可以准确有效地找到看不见的对象和部分视图RGBD图像之间的相应点。然后，它使用算法鲁棒到对象对称性从对应关系中计算6D姿势。广泛的实验表明，我们的方法的表现优于几个直观基线，从而验证其有效性。所有数据，代码和模型都将公开可用。项目页面：www.graspnet.net/unseen6d

最近，事物的人工智能（Aiot）一直在引起人们的关注，具有通过事物的网络连接提供高度智能服务的有趣愿景，从而导致了先进的AI驱动生态。但是，对数据隐私的最新监管限制排除将敏感的本地数据上传到数据中心，并以集中式方法利用它们。在这种情况下，直接应用联合学习算法几乎不能满足效率和准确性的工业要求。因此，我们在面部识别应用方面为AIOT提出了一个有效的工业联合学习框架。具体而言，我们建议利用转移学习的概念来加快设备上的联合培训，并进一步介绍私人投影仪的新颖设计，该设计有助于保护共享梯度，而不会产生额外的记忆消耗或计算成本。对亚洲私人面部数据集的实证研究表明，我们的方法仅在20轮沟通中就可以实现高认识的准确性，这表明了其在预测和培训方面的有效性。

我们介绍了一个大规模实验，该实验对编码器进行了预处理，其参数计数范围从700m到9.3b不等，随后蒸馏到较小的型号中，范围为17m-170亿参数，其应用到自然语言理解（NLU）组件（NLU）组件（虚拟助手系统。尽管我们使用70％的口语数据训练，但在对书面形式的跨语性自然语言推论（XNLI）语料库进行评估时，我们的教师模型与XLM-R和MT5相当。我们使用系统中的内域数据对教师模型进行了第二阶段的训练，以提高了3.86％的相对分类，而相对7.01％的插槽填充。我们发现，即使是从我们的2阶段教师模型中提取的170亿参数模型，与仅接受公共数据的2.3B参数老师相比，与2.3B参数老师相比，意图分类更好2.88％，并且7.69％的插槽填充错误率更好（第1阶段），强调了。内域数据对训练的重要性。当使用标记的NLU数据进行离线评估时，我们的17m参数阶段2蒸馏模型的表现分别优于XLM-R碱基（85m Params）和Distillbert（42m Params），分别优于4.23％至6.14％。最后，我们介绍了一个完整的虚拟助手实验平台的结果，在该平台中，我们发现使用经过预训练和蒸馏管道训练的模型超过了从8500万参数教师蒸馏的模型，在自动测量全系统用户不满的自动测量中，从8500万参数教师蒸馏出3.74％-4.91％。

基于自我注意力的模型，例如视觉变压器（VIT），已经成为计算机视觉中卷积神经网络（CNN）的一种非常有竞争力的建筑。尽管越来越高的变体具有更高的识别精度，但由于自我注意力的二次复杂性，现有的VIT通常在计算和模型大小中要求。尽管已重新引入了最近的CNN的几种成功设计选择（例如，卷积和分层多阶段结构）已重新引入最近的VIT，但它们仍然不足以满足移动设备的有限资源要求。这激发了最近根据最先进的Mobilenet-V2开发光线的尝试，但仍然留下了性能差距。在这项工作中，在这个研究不足的方向上进一步推动了Edgevits，这是一个新的轻巧vits家族，这首先使基于注意力的视觉模型能够与最佳轻巧的CNN竞争，这准确性和设备效率。这是通过基于自我注意力和卷积的最佳整合而引入高度成本效益的本地 - 全球局（LGL）信息交换瓶颈来实现的。对于设备青年的评估，我们不再依赖诸如拖船或参数的不准确代理，而是采用一种实用的方法来直接专注于设备延迟，以及首次首次提供能源效率。具体而言，我们表明，当考虑准确性的延迟和准确性 - 能量折衷时，我们的模型是帕累托最佳的，在几乎所有情况下都严格占据了其他VIT并与最有效的CNN竞争的严格优势。代码可从https://github.com/saic-fi/edgevit获得。

在本文中，我们介绍了VCSL（视频复制段本地化），这是一种新的综合段级注释的视频复制数据集。与受视频级注释或小规模限制的现有复制检测数据集相比，VCSL不仅具有两个段级标签的数据级，其中有160k现实的视频副本对，其中包含超过280k的本地化copied seggment对，而且还包含超过280k涵盖各种视频类别和各种视频持续时间。每个收集的视频对中的所有复制段均经过手动提取，并伴随着精确注释的启动和结束时间戳。除了数据集外，我们还提出了一种新颖的评估协议，该协议可以更好地衡量视频对之间复制重叠段的预测准确性，并在不同情况下显示出改善的适应性。通过使用拟议的数据集和评估指标对几个基线和最先进的细分级视频副本检测方法进行基准测试，我们提供了一项全面的分析，可以揭示当前方法的优势和劣势作品。 VCSL数据集，公制和基准代码均在https://github.com/alipay/vcsl上公开获得。

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

In this paper, we study the problem of knowledge-intensive text-to-SQL, in which domain knowledge is necessary to parse expert questions into SQL queries over domain-specific tables. We formalize this scenario by building a new Chinese benchmark KnowSQL consisting of domain-specific questions covering various domains. We then address this problem by presenting formulaic knowledge, rather than by annotating additional data examples. More concretely, we construct a formulaic knowledge bank as a domain knowledge base and propose a framework (ReGrouP) to leverage this formulaic knowledge during parsing. Experiments using ReGrouP demonstrate a significant 28.2% improvement overall on KnowSQL.

Although many studies have successfully applied transfer learning to medical image segmentation, very few of them have investigated the selection strategy when multiple source tasks are available for transfer. In this paper, we propose a prior knowledge guided and transferability based framework to select the best source tasks among a collection of brain image segmentation tasks, to improve the transfer learning performance on the given target task. The framework consists of modality analysis, RoI (region of interest) analysis, and transferability estimation, such that the source task selection can be refined step by step. Specifically, we adapt the state-of-the-art analytical transferability estimation metrics to medical image segmentation tasks and further show that their performance can be significantly boosted by filtering candidate source tasks based on modality and RoI characteristics. Our experiments on brain matter, brain tumor, and white matter hyperintensities segmentation datasets reveal that transferring from different tasks under the same modality is often more successful than transferring from the same task under different modalities. Furthermore, within the same modality, transferring from the source task that has stronger RoI shape similarity with the target task can significantly improve the final transfer performance. And such similarity can be captured using the Structural Similarity index in the label space.